Implementations of the claimed invention generally may relate to processing graphical images and, more particularly, to processing graphical images using geometry images.
FIG. 1 is a flow chart illustrating typical graphics processing 100. Geometry representing graphical images may be submitted to a graphics pipeline [act 110]. Such geometry typically may include triangles, but may include quadrilaterals or any other type of geometry. Vertex shaders may then process the geometry, performing such processing as transforming and lighting each polygon, determining colors for vertices, determining normals if necessary, etc. [act 120]. Vertex shaders typically include hardware that implements an instruction set to operate on data associated with vertices. Such data may include the vertex position, vertex normal, and per vertex texture coordinates. These values may also be used later by the shading hardware to interpolate across a polygon to create a shaded image.
Next, the polygons may be clipped to the screen in appropriate positions [act 130]. Pixel shaders may then shade each polygon [act 140]. Pixel shaders typically include hardware that execute a set of instructions on each fragment as it passes through the graphics pipeline before being rendered to the screen. Pixel and vertex shaders may have identical or different instruction sets. Additionally, the instruction sets they execute may be different than the instructions exposed to a programmer.
After shading, if a particular pixel is in front of any previously rendered pixels, it may be written to the frame buffer [act 150]. In some graphical engines, alpha blending may occur. Alpha blending is a mechanism to facilitate the implementation of partially transparent objects and typically may be realized through an additional channel to the color channels used in a traditional three-dimensional (3D) graphics pipeline.
Recently, a technique of representing the surface geometry of graphical images has been proposed that may have certain advantages. Such a technique may remesh an arbitrary graphical surface onto a completely regular structure called a “geometry image.” The geometry image may capture the surface geometry of a graphical image as a two-dimensional (2D) array of quantized points. Surface signals, such as normals and/or colors, may be stored in similar 2D images using the same surface parameterization as the geometry image. Further discussion of geometry images may be found in, for example, Xianfeng Gu et al., “Geometry Images,” SIGGRAPH 2002 Proceedings, pp. 355-361; and F. Losasso et al., “Smooth Geometry Images,” Eurographics 2003, pp. 138-145 and 273. To better understand what a geometry image is and how it may represent a 3D graphical object, a brief explanation will be provided.
FIG. 2 illustrates formation of a geometry image 250 from a 3D model 210 (see, e.g., Hughes Hoppe, “Irregular To Completely Regular Meshing In Computer Graphics,” International Meshing Roundtable 2002, September 2002). 3D model 210 may be conceptualized as a two-dimensional (2D) shape bent and warped in 3D space to form a 3D mesh. Model 210 may include such a mesh of geometric shapes (e.g., triangles, quadrilaterals, etc.) that are defined by vertices and edges between the vertices. The vertices in model 210 may have associated spatial coordinate values in a 3D coordinate system (e.g., [x,y,z] coordinates). 3D Model 210 may be cut along its edges to form a different (e.g., non-rabbit-shaped) 2D shape. Cut 220 illustrates one such cut. This different shape may be warped or parameterized using known techniques to form a regular shape 230. Regular shape 230 may include connected vertices from model 210 (e.g., illustrated as triangles), with the cut 220 being located at the outer edge(s) of shape 230. Thus, regular shape 230 retains both the [x,y,z] position values of the vertices in model 210, as well as the connection relationships between these vertices. It should be noted that although regular shape 230 is illustrated as a square, other types of regular shapes (e.g., circles, etc.) may also be used.
The polygons in regular shape 230 may be regularly sampled to generate sampled shape 240. Each sampled point in sampled shape 240 may be a new vertex that both retains a positional value (e.g., an [x,y,z] coordinate) and that is implicitly connected to other vertices in sampled image 240. That is, vertices in sampled image 240 may be connected be edges to other vertices to their top, bottom, left, and right sides by virtue of the regular sampling of regular shape 230.
Sampled image 240 may be converted into a red/green/blue (RGB) (or any other color space) color image by mapping the three-channel [x,y,z] coordinates of the vertices in sampled image 240 to the three-channel RGB space, [r,g,b] coordinates, to produce geometry image 250. Each pixel in geometry image 250 may represent a sampled vertex in model 210, with the red, green, and blue values of the pixel representing the [x,y,z] spatial position of the sampled vertex. The neighbor vertices of the corresponding sampled vertex in model 210 are determined by the neighboring pixels to a given pixel in geometry image 250. In other words, the neighbor vertices of model 210 are known by the neighbor pixels in geometry image 250, and the positional values of these vertices are known by the color values for pixels in geometry image 250.
Although geometry image 250 has been explained with regard to positional information, geometry images 250 may be generated for any information associated with the vertices of model 210. For example, normals, which are [x,y,z] vectors that point into space, may also be represented by a geometry image 250. Also, color values and/or texture coordinates may also be represented by geometry images 250. Representing models as images may allow one to use existing image processing algorithms, such as image compression algorithms.
Although it is possible to compress geometry images, such geometry images, even if compressed, may in some instances be sent to a graphics processor over a data bus. In some cases, the bandwidth of the data bus may limit the resolution and/or visual fidelity able to be displayed by the graphics processor from the geometry images.